A new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains

Continuous-time homogeneous Markov chains

Taylor Expansion for the Entropy Rate of Hidden Markov Chains

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...

The Rate of Rényi Entropy for Irreducible Markov Chains

In this paper, we obtain the Rényi entropy rate for irreducible-aperiodic Markov chains with countable state space, using the theory of countable nonnegative matrices. We also obtain the bound for the rate of Rényi entropy of an irreducible Markov chain. Finally, we show that the bound for the Rényi entropy rate is the Shannon entropy rate.

On nonlinear discrete-time systems driven by Markov chains

The behavior of a class of hybrid systems in discrete-time can be represented by nonlinear difference equations with a Markov input. The analysis of such a system usually starts by establishing the Markov property of the joint process formed by combining the system’s state and input. There are, however, no complete proofs of this property. This paper aims to address this problem by presenting a...

Hierarchical Counterexamples for Discrete-Time Markov Chains

This paper introduces a novel counterexample generation approach for the verification of discrete-time Markov chains (DTMCs) with two main advantages: (1) We generate abstract counterexamples which can be refined in a hierarchical manner. (2) We aim at minimizing the number of states involved in the counterexamples, and compute a critical subsystem of the DTMC whose paths form a counterexample....